Empowering Automatic Decision-Making Systems: General Intelligence, Responsibility, and Moral Sensibility

0. Introduction Before a human being begins to make decisions and take unsupervised actions in a professional capacity which significantly affect others, s/he is usually explicitly empowered to do so, by means of some socially and/or legally sanctioned process of training and evaluation. At a time when it is being suggested that computational artifacts may take up roles with significant human impact, ranging from medical diagnosis to automatic launch on warning of nuclear missiles, it becomes appropriate to ask whether sufficient thought has been given to the question of establishing empowerment processes for such systems if they are to act autonomously without human supervision. I believe that a careful and responsible investigation of this question wil l lead to a paradox that the sorts of special-purpose, focussed systems which we can imagine being within reach technically wil l be manifestly and necessarily incapable of satisfying certain necessary criteria for empowerment, despite our inability to objectively define such criteria or design explicit tests to implement them. And this inability wil l in turn frustrate us if in the unforeseeably distant future we are finally in a position to build general-purpose, broadly intelligent systems*. In what follows I consider first the proximate form of the paradox, as it applies to special-purpose systems, and then the longer term, more general case. The treatment is, given the constraints of space, time and the author's expertise, necessarily incomplete and anecdotal, rather than exhaustive and authoritative, but may at least serve to provoke debate.